Markov decision processes with recursive risk measures
نویسندگان
چکیده
In this paper, we consider risk-sensitive Markov Decision Processes (MDPs) with Borel state and action spaces unbounded cost under both finite infinite planning horizons. Our optimality criterion is based on the recursive application of static risk measures. This motivated by utilities in economic literature, has been studied before for entropic measure extended here to an axiomatic characterization suitable We derive a Bellman equation prove existence Markovian optimal policies. For horizon, model shown be contractive policy stationary. Moreover, establish connection distributionally robust MDPs, which provides global interpretation recursively defined objective function. Monotone models are particular.
منابع مشابه
Verification of Markov Decision Processes with Risk-Sensitive Measures
We develop a method for computing policies in Markov decision processes with risk-sensitive measures subject to temporal logic constraints. Specifically, we use a particular risk-sensitive measure from cumulative prospect theory, which has been previously adopted in psychology and economics. The nonlinear transformation of the probabilities and utility functions yields a nonlinear programming p...
متن کاملReachability in Recursive Markov Decision Processes
We consider a class of infinite-state Markov decision processes generated by stateless pushdown automata. This class corresponds to 112 -player games over graphs generated by BPA systems or (equivalently) 1-exit recursive state machines. An extended reachability objective is specified by two sets S and T of safe and terminal stack configurations, where the membership to S and T depends just on ...
متن کاملIterated risk measures for risk-sensitive Markov decision processes with discounted cost
We demonstrate a limitation of discounted expected utility, a standard approach for representing the preference to risk when future cost is discounted. Specifically, we provide an example of the preference of a decision maker that appears to be rational but cannot be represented with any discounted expected utility. A straightforward modification to discounted expected utility leads to inconsis...
متن کاملMore Risk-Sensitive Markov Decision Processes
We investigate the problem of minimizing a certainty equivalent of the total or discounted cost over a finite and an infinite horizon which is generated by a Markov Decision Process (MDP). The certainty equivalent is defined by U−1(EU(Y )) where U is an increasing function. In contrast to a risk-neutral decision maker this optimization criterion takes the variability of the cost into account. I...
متن کاملRisk-Sensitive Control of Markov Decision Processes
This paper introduces an algorithm to determine near-optimal control laws for Markov Decision Processes with a risk-sensitive criterion. Both the fully observed and the partially observed settings are considered, for nite and innnite horizon formulations. Dynamic programming equations are introduced which characterize the value function for the partially observed, innnite horizon , discounted c...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: European Journal of Operational Research
سال: 2022
ISSN: ['1872-6860', '0377-2217']
DOI: https://doi.org/10.1016/j.ejor.2021.04.030